Assist device, endoscope system, assist method and computer-readable recording medium

ABSTRACT

An assist device includes: a processor configured to generate a fluorescence image based on an imaging signal that is generated by capturing an image of fluorescence caused by excitation light that is applied to a living tissue, calculate a fluorescence intensity based on the fluorescence image, based on the fluorescence intensity, estimate a placement period during which a medical tool is placed in a lumen, and output placement period information on the placement period and an observation image obtained by capturing an image of the living tissue.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/JP2020/036945, filed on Sep. 29, 2020, the entire contents of whichare incorporated herein by reference.

BACKGROUND 1. Technical Field

The disclosure relates to an assist device for determining a placementperiod during which a stent is placed in a urinary tract, and anendoscope system, an assist method and a computer-readable recordingmedium.

2. Related Art

As for endoscopes, a technique of applying a laser beam to a calculusthat is caused in a urinary tract to fragment the calculus has beenknown (refer to, for example, Japanese National Publication ofInternational Patent Application No. 2017-500172). According to thetechnique, when it is confirmed that an aiming beam is incident on atarget lump of calculus, or the like, an energy source is operated toapply an energy pulse onto the target lump via an energy guide. In thiscase, because there is a possibility that the ureter may be damaged, astent is placed to protect the ureter after fragmentation of calculus(refer to, for example, Japanese National Publication of InternationalPatent Application No. 2017-510371).

SUMMARY

In some embodiments, an assist device includes: a processor configuredto generate a fluorescence image based on an imaging signal that isgenerated by capturing an image of fluorescence caused by excitationlight that is applied to a living tissue, calculate a fluorescenceintensity based on the fluorescence image, based on the fluorescenceintensity, estimate a placement period during which a medical tool isplaced in a lumen, and output placement period information on theplacement period and an observation image obtained by capturing an imageof the living tissue.

In some embodiments, an endoscope system includes: an endoscope that isinsertable into a lumen of a subject; a light source configured to emitexcitation light that excites advanced glycation end products caused byperforming thermal treatment on a living tissue; and a control devicethat is detachable from the endoscope, the endoscope including animaging device configured to generate an imaging signal by capturing animage of fluorescence caused by the excitation light, and a cut filterthat is provided on a side of a light receiving surface of the imagingdevice, the cut filter being configured to block light on a side ofshort wavelengths containing part of a wavelength band of the excitationlight, the control device including an assist device that assists apractitioner, and the assist devise including a processor configured togenerate a fluorescence image based on the imaging signal, calculate afluorescence intensity based on the fluorescence image, based on thefluorescence intensity, estimate a placement period during which amedical tool is placed in the lumen, and output placement periodinformation on the placement period and an observation image obtained bycapturing an image of the living tissue.

In some embodiments, provided is an assist method executed by an assistdevice. The method includes: generating a fluorescence image based on animaging signal that is generated by capturing an image of fluorescencecaused by excitation light that is applied to a living tissue;calculating a fluorescence intensity based on the fluorescence image;based on the fluorescence intensity, estimating a placement periodduring which a medical tool is placed in a lumen; and outputtingplacement period information on the placement period and an observationimage obtained by capturing an image of the living tissue.

In some embodiments, provided is a non-transitory computer-readablerecording medium with an executable program stored thereon. The programcauses an assist device to execute generating a fluorescence image basedon an imaging signal that is generated by capturing an image offluorescence caused by excitation light that is applied to a livingtissue; calculating a fluorescence intensity based on the fluorescenceimage; based on the fluorescence intensity, estimating a placementperiod during which a medical tool is placed in a lumen; and outputtingplacement period information on the placement period and an observationimage obtained by capturing an image of the living tissue.

The above and other features, advantages and technical and industrialsignificance of this disclosure will be better understood by reading thefollowing detailed description of presently preferred embodiments of thedisclosure, when considered in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically illustrating an entire configurationof an endoscope system according to an embodiment;

FIG. 2 is a block diagram illustrating a functional configuration of arelevant part of the endoscope system according to the embodiment;

FIG. 3 is a diagram schematically illustrating wavelengthcharacteristics of excitation light that a second light source unitemits according to the embodiment;

FIG. 4 is a diagram schematically illustrating transmissioncharacteristics of a cut filter according to the embodiment;

FIG. 5 is a diagram illustrating an example of correlation informationthat a correlation information recorder records according to theembodiment;

FIG. 6 is a diagram schematically illustrating a principle ofobservation in a fluorescence observation mode of the endoscope systemaccording to the embodiment;

FIG. 7 is a diagram schematically illustrating a principle ofobservation in a normal observation mode of the endoscope systemaccording to the embodiment;

FIG. 8 is a flowchart illustrating a manipulation method of flexibletransurethral lithotomy performed by a practitioner using the endoscopesystem;

FIG. 9A is a diagram illustrating an example of transition in an imagethat a display device displays in flexible transurethral lithotomy;

FIG. 9B is a diagram illustrating an example of the transition in theimage that the display device displays in flexible transurethrallithotomy;

FIG. 9C is a diagram illustrating an example of the transition in theimage that the display device displays in flexible transurethrallithotomy;

FIG. 9D is a diagram illustrating an example of the transition in theimage that the display device displays in flexible transurethrallithotomy;

FIG. 9E is a diagram illustrating an example of the transition in theimage that the display device displays in flexible transurethrallithotomy;

FIG. 9F is a diagram illustrating an example of the transition in theimage that the display device displays in flexible transurethrallithotomy;

FIG. 10 is a flowchart illustrating an overview of a process that anendoscope system according to the embodiment executes;

FIG. 11 is a diagram schematically illustrating a method of making anestimation on whether to place a stent that is performed by anestimation unit of the endoscope system according to the embodiment;

FIG. 12 is a diagram schematically illustrating a method of, by theestimator of the endoscope system, estimating a period during which astent is placed according to the embodiment;

FIG. 13 is a diagram illustrating an example of an output screen that anoutput unit of the endoscope system outputs to the display deviceaccording to the embodiment;

FIG. 14 is a diagram illustrating another example of the output screenthat the output unit of the endoscope system outputs to the displaydevice according to the embodiment;

FIG. 15 is a diagram illustrating another example of the output screenthat the output unit of the endoscope system outputs to the displaydevice according to the embodiment;

FIG. 16 is a diagram illustrating another example of the output screenthat the output unit of the endoscope system outputs to the displaydevice according to the embodiment; and

FIG. 17 is a diagram schematically illustrating transmissioncharacteristics of a cut filter according to a modification of theembodiment.

DETAILED DESCRIPTION

An endoscope system using a flexible-scope that is used fortransurethral lithotomy (“TUL” below) will be described below as a modefor carrying out the disclosure (“embodiment” below); however,embodiments are not limited thereto. For example, a rigid scope and asurgical robot, or the like, are usable. The embodiment does not limitthe disclosure. Furthermore, as for illustration of the drawings, thesame components are denoted with the same reference numerals anddescribed. Furthermore, it is necessary to note that the drawings areschematic and the relationship between the thickness and width of eachmember, the proportion of each member, etc., are different from actualones. Portions different in mutual sizes and proportions between thedrawings may be contained as well.

Configuration of Endoscope System

FIG. 1 is a diagram schematically illustrating an entire configurationof an endoscope system according to an embodiment. An endoscope system 1illustrated in FIG. 1 captures an internal image of the body of asubject, such as a patient, by inserting an insertion portion of anendoscope into a body cavity or a lumen of the subject, for example,into the urinary tract and displays a display image based on an imagingsignal of the captured image on a display device. The urinary tractincludes the urethra, the bladder, the ureter and the kidney and has aductal form extending in a depth direction. A practitioner, such as adoctor, fragments a calculus in the subject with a laser irradiationdevice that applies a high-power infrared laser, such as a holmium YAGlaser, via the endoscope while observing the display image that thedisplay device displays, extracts the fragmented calculus with atreatment tool, such as a basket catheter, and places a medical tool inthe urinary tract for a given period. The medical tool is any one of astent, a catheter, and an indwelling needle. The endoscope system 1includes an endoscope 2, a display device 3, a control device 4, and alaser irradiation device 5.

Configuration of Endoscope

First of all, a configuration of the endoscope 2 will be described.

The endoscope 2 generates an imaging signal (RAW data) of a capturedinternal image of the body of the subject and outputs the generatedimaging signal to the control device 4. The endoscope 2 includes aninsertion portion 21, an operation unit 22, and a universal cord 23.

The insertion portion 21 is inserted into the subject. The insertionportion 21 is flexible and elongated. The insertion portion 21 includesa distal end part 24 that incorporates an imaging device to be describedbelow, a curve part 25 that is formed of multiple curve pieces and thatflexibly curves, and a flexible tube 26 that is flexible and elongatedand that is connected to a proximal end side of the curve part 25.

The distal end part 24 is configured using glass fibers, etc. The distalend part 24 forms a light guide path for illumination light that issupplied from the control device 4 via the universal cord 23 and theoperation unit 22, generates an imaging signal of a captured image ofreturn light of the illumination light, and outputs the imaging signalto the control device 4.

The operation unit 22 includes a curve knob 221 that causes the curveunit 25 to curve in up and down directions and left and rightdirections, a treatment tool insertion port 222 into which a treatmenttool is inserted, and a plurality of switches 223 serving as anoperation input unit that, in addition to the control device 4, inputsoperation instruction signals to peripherals, such as an air supplyunit, a water supply unit and a gas supply unit, a pre-freeze signal ofan instruction for the endoscope system 1 to capture a still image, or aswitch signal that switches an observation mode of the endoscope system1. The treatment tool that is inserted from the treatment tool insertionportion 222 goes out of an opening (not illustrated in the drawing) viaa treatment tool channel (not illustrated in the drawing) of the distalend part 24. The treatment tool is the laser irradiation device 5, thebasket catheter, or the like.

The universal cord 23 incorporates at least a light guide and anassembly cable including a single cable or a bundle of cables. Theassembly cable includes a signal line for transmitting and receiving asignal between the endoscope 2 and the control device 4 and fortransmitting and receiving the imaging signal (RAW data) and a signalline for transmitting and receiving a drive timing signal (asynchronization signal and a clock signal) for driving the imagingdevice to be described below. The universal cord 23 includes a connector27 that is detachable from the control device 4 and a connector 28 towhich a coiled coil cable 27 a extends and that is detachable from thecontrol device 4 at an end of extension of the coil cable 27 a.

Configuration of Display Device

A configuration of the display device 3 will be described next.

The display device 3 displays a display image based on a video signalthat is input from the control device 4 under the control of the controldevice 4. The display device 3 is realized using a display panel oforganic electro luminescence (EL), liquid crystals, or the like.

Configuration of Control Device

A configuration of the control device 4 will be described next.

The control device 4 controls each unit of the endoscope system 1. Thecontrol device 4 supplies illumination light to be applied to thesubject by the endoscope 2. The control device 4 performs various typesof image processing on the imaging signal that is input from theendoscope 2 and outputs the processed imaging signal to the displaydevice 3.

Configuration of Laser Irradiation Device

A configuration of the laser irradiation device will be described next.

The laser irradiation device 5 is inserted into the body of the subject,for example, into the urinary tract (for example, the kidney, theureter, the bladder and the urethra) via the treatment tool insertionportion 222 of the endoscope 2 and, under the operation of thepractitioner, applies a high-power infrared laser, such as a holmium YAGlaser, to a calculus caused in the subject, thereby fragmenting thecalculus.

Functional Configuration of Relevant Part of Endoscope System

A functional configuration of a relevant part of the endoscope system 1described above will be described next. FIG. 2 is a block diagramillustrating the functional configuration of the relevant part of theendoscope system 1.

Configuration of Endoscope

First of all, a configuration of the endoscope 2 will be described.

The endoscope 2 includes an illuminating optical system 201, an imagingoptical system 202, a cut filter 203, an imaging device 204, an A/Dconverter 205, a P/S converter 206, an imaging recorder 207, and animaging controller 208. Note that each of the illuminating opticalsystem 201, the imaging optical system 202, the cut filter 203, theimaging device 204, the A/D converter 205, the P/S converter 206, theimaging recorder 207, and the imaging controller 208 is arranged in thedistal end part 24.

The illuminating optical system 201 applies the illumination light thatis supplied from a light guide 231 that is formed of optical fibers,etc., to the subject (living tissue). The illuminating optical system201 is realized using a single lens, a plurality of lenses, or the like.

The imaging optical system 202 focuses light, such as reflection lightthat is reflected from the subject, return light from the subject, orfluorescence that the subject emits, thereby forming a subject image(ray of light) on a light receiving surface of the imaging device 204.The imaging optical system 202 is realized using a single lens, aplurality of lenses, or the like.

The cut filter 203 is arranged on an optical axis O1 of the imagingoptical system 202 and the imaging device 204. The cut filter 203 blockslight having a wavelength band of reflection light or return light ofthe excitation light that is supplied from the control device 4 to bedescribed below and that is from the subject and transmits light havingthe wavelength band on a side of wavelengths longer than those of theexcitation light. The transmission characteristics of the cut filter 203will be described below.

Under the control of the imaging controller 208, the imaging device 204receives the subject image (ray of light) that is formed by the imagingoptical system 202 and that is transmitted through the cut filter 203,performs photoelectric conversion to generate imaging signal (RAW data),and outputs the imaging signal to the A/D converter 205. The imagingdevice 204 is realized using a charge coupled device (CCD) orcomplementary metal oxide semiconductor (CMOS) image sensor that isformed by arranging any one of color filters forming a Bayer array(RGGB) in each of a plurality of pixels that are formed by beingarranged in a two-dimensional matrix.

Under the control of the imaging controller 208, the A/D converter 205performs the A/D conversion processing on the analog imaging signal thatis input from the imaging device 204 and outputs the processed imagingsignal to the P/S converter 206. The A/D converter 205 is realized usingan A/D conversion circuit, or the like.

Under the control of the imaging controller 208, the P/S converter 206performs parallel/serial conversion on the digital imaging signal thatis input from the A/D converter 205 and outputs the imaging signal onwhich the parallel/serial conversion has been performed to the controldevice 4 via a first transmission cable 232. The P/S converter 206 isrealized using the P/S conversion circuit, or the like. Note that,according to the first embodiment, an E/O converter that converts animaging signal into an optical signal may be provided instead of the P/Sconverter 206 and the imaging signal may be output to the control device4 using an optical signal and may be transmitted to the control device 4by wireless communication according to, for example, Wi-Fi (wirelessfidelity (trademark).

The imaging recorder 207 records various types of information on theendoscope 2 (for example, pixel information on the imaging device 204and the characteristics of the cut filter 203). The imaging recorder 207records various types of setting data and parameters for control thatare transmitted from the control device 4 via a second transmissioncable 233. The imaging recorder 207 is configured using a non-volatilememory or a volatile memory.

The imaging controller 208 controls operations of each of the imagingdevice 204, the A/D converter 205, and the P/S converter 206 based onthe setting data that is received from the control device 4 via thesecond transmission cable 233. The imaging controller 208 is realizedusing a time generator (TG), a processor that is a processing deviceincluding hardware, such as a CPU, and a memory that is a temporarystorage area that the processor uses.

Configuration of Control Device

A configuration of the control device 4 will be described next.

The control device 4 includes a condenser lens 401, a first light sourceunit 402, a second light source unit 403, a light source controller 404,a S/P converter 405, an image processor 406, an input unit 407, arecorder 408, and a controller 409.

The condenser lens 401 focuses light that is emitted by each of thefirst light source unit 402 and the second light source unit 403 andemits the light to the light guide 231. The condenser lens 401 isconfigured using a single lens or a plurality of lenses.

Under the control of the light source controller 404, the first lightsource unit 402 emits white light (normal light) that is visible light,thereby supplying white light to the light guide 231. The first lightsource unit 402 is configured using a white light emitting diode (LED)lamp, a driver, etc. The first light source unit 402 may simultaneouslyemit light with a red LED lamp, a green LED lamp, and a blue LED lamp,thereby supplying white light that is visible light. Needless to say,the first light source unit 402 may be configured using a halogen lampor a xenon lamp.

Under the control of the light source controller 404, the second lightsource unit 403 emits excitation light having a given wavelength band,thereby supplying a narrow-band light as the illumination light to thelight guide 231. The excitation light has a wavelength band from 400nanometers (nm) to 430 nm (the center wavelength is 415 nm). The secondlight source unit 403 is realized using a collimating lens, asemiconductor laser, such as a violet laser diode (LD), a driver, etc.The wavelength characteristics of each of the white light that isemitted by the first light source unit 402 and the excitation light thatis emitted by the second light source unit 403 will be described below.

The light source controller 404 is configured using a processor that isa processing device including hardware, such as a field programmableagate array (FPGA) or a CPU, and a memory that is a temporary storagearea that the processor uses. The light source controller 404 controlslight emission timing, the light emission intensity and the lightemission time, etc., of each of the first light source unit 402 and thesecond light source unit 403.

Under the control of the controller 409, the S/P converter 405 performsserial/parallel conversion on the imaging signal that is received fromthe endoscope 2 via the first transmission cable 232 and outputs theprocessed imaging signal to the image processor 406. Note that, in thecase where the endoscope 2 outputs the imaging signal in an opticalsignal, an O/E converter that converts the optical signal into anelectric signal may be provided instead of the S/P converter 405. In thecase where the endoscope 2 transmits the imaging signal by wirelesscommunication, a communication module capable of receiving a radiosignal may be provided instead of the S/P converter 405.

The image processor 406 is realized using a processor includinghardware, such as a CPU, a graphics processing unit (GPU) or a FPGA, anda memory that is a temporary storage area that the processor uses. Underthe control of the controller 409, the image processor 406 performsgiven image processing on the imaging signal that is input from the S/Pconverter 405 and outputs the processed imaging signal to the displaydevice 3. Note that, in the embodiment, the image processor 406functions as an assist device. The image processor 406 includes agenerator 406 a, a calculator 406 c, an extractor 406 b, an estimator406 d, and an output unit 406 e.

The generator 406 a generates a fluorescence image based on the imagingsignal that is generated by capturing an image of fluorescence caused bythe excitation light that is applied to living tissue. Specifically, thegenerator 406 a acquires the imaging signal from the imaging device 204of the endoscope 2 via the A/D converter 205, the P/S converter 206, thefirst transmission cable 232, and the S/P converter 405 (simplydescribed as “acquires from the imaging device 204 of the endoscope 2”below). The generator 406 a generates a fluorescence image based on theimaging signal that is acquired from the imaging device 204 of theendoscope 2 and that is generated by capturing an image of thefluorescence caused by the excitation light that is applied to theliving tissue. The generator 406 a generates an observation image (whitelight image) that is a display image based on an imaging signal that isgenerated by imaging the reflection light that is reflected from theliving tissue because of application of white light to the living tissueand the return light.

The extractor 406 b extracts a fluorescence area from the fluorescenceimage. Specifically, the extractor 406 b extracts a fluorescence area byperforming a binary process on each pixel of the fluorescence image. Forexample, the extractor 406 b extracts a fluorescence area by extractingpixels of the fluorescence image whose pixel values are equal to orlarger than a given value.

The calculator 406 c calculates a fluorescence intensity based on thefluorescence image that is generated by the generator 406 a.Specifically, the calculator 406 c calculates a fluorescence intensityof the fluorescence area that is the fluorescence image generated by thegenerator 406 a and that is extracted by the extractor 406 b.

The estimator 406 d estimates a placement period during which a medicaltool is placed in a lumen based on the fluorescence intensity that iscalculated by the calculator 406 c. The lumen is the urinary tract here.The urinary tract includes the urethra, the bladder, the ureter and thekidney. The medical tool is any one of a stent, a catheter, and anindwelling needle. The estimator 406 d estimates a degree of invasion ofliving tissue with an energy device based on the fluorescence intensitythat is calculated by the calculator 406 c and, based on the degree ofinvasion, estimates the period during which the medical tool is placed.Specifically, the estimator 406 d estimates a degree of invasion ofliving tissue with the energy device based on correlation informationrepresenting a correlation between the degree of invasion and thefluorescence intensity that a correlation information recorder 408 b tobe described below records and the fluorescence intensity that iscalculated by the calculator 406 c.

The output unit 406 e outputs placement period information on theplacement period for the medical tool that is estimated by the estimator406 d and the observation image serving as the display image obtained bycapturing an image of the living tissue to the display device 3.

The input unit 407 receives inputs of various types of operations on theendoscope system 1 and outputs the received operations to the controller409. The input unit 407 is configured using a mouse, a foot switch, akeyboard, a button, a switch, a touch panel, etc.

The recorder 408 is realized using a volatile memory, a non-volatilememory, a solid state drive (SSD) or a hard disk drive (HDD) or arecording medium, such as a memory card. The recorder 408 records datacontaining various types of parameters necessary for operations of theendoscope system 1. The recorder 408 includes a program recorder 408 athat records various types of programs for running the endoscope system1 and the correlation information recorder 408 b.

The correlation information recorder 408 b records the degree ofinvasion of the living tissue of the subject with the laser irradiationdevice 5 and the intensity of fluorescence that is emitted whenexcitation light is applied to the living tissue that is thermallytreated by the laser irradiation device 5. Details of the correlationinformation will be described below.

The controller 409 is realized using a processor including hardware,such as a FPGA or a CPU, and a memory that is a temporary storage areathat the processor uses. The controller 409 generally controls each ofthe units forming the endoscope system 1.

Wavelength Characteristics of Excitation Light

Wavelength characteristics of excitation light that is emitted by thesecond light source unit 403 will be described next.

FIG. 3 is a diagram schematically illustrating the wavelengthcharacteristics of the excitation light that the second light sourceunit 403 emits. In FIG. 3 , the horizontal axis represents thewavelength (nm) and the vertical axis represents the wavelengthcharacteristics. In FIG. 3 , the polygonal chain L_(V) represents thewavelength characteristic of the excitation light that is emitted by thesecond light source unit 403. In FIG. 3 , the curve L_(B) represents thewavelength band of blue, the curve L_(G) represents the wavelength bandof green, and the curve L_(R) represents the wavelength band of red.

As illustrated in FIG. 3 , the second light source unit 403 emits theexcitation light having a center wavelength (peak wavelength) of 415 nmand a wavelength band from 400 to 430 nm.

Transmission Characteristics of Cut Filter

The transmission characteristics of the cut filter 203 will be describednext.

FIG. 4 is a diagram schematically illustrating the transmissioncharacteristics of the cut filter 203. In FIG. 4 , the horizontal axisrepresents the wavelength (nm) and the vertical axis represents thetransmission characteristics. In FIG. 4 , the polygonal chain L_(F)represents the transmission characteristic of the cut filter 203, thepolygonal chain L_(V) represents the wavelength characteristics of theexcitation light, and the polygonal chain L_(NG) represents thewavelength characteristics of fluorescence caused by application of theexcitation light to advanced glycation end products caused by thermaltreatment on living tissue performed by an energy device, for example,the laser irradiation device 5.

As represented by the polygonal chain L_(V) and the polygonal chainL_(NG), the cut filter 203 blocks part of the excitation light that isreflected from the living tissue of the observation area and transmitslight having another wavelength band containing fluorescence components.Specifically, the cut filter 203 blocks part of the light having awavelength band on a short-wavelength side from 400 nm and under 430 nmcontaining the excitation light and transmits light having a wavelengthband on a long wavelength side from 430 nm containing fluorescencecaused by application of the excitation light to the advanced glycationend products caused by the thermal treatment.

Correlation Information

An example of the correlation information that the correlationinformation recorder 408 b records will be described next.

FIG. 5 is a diagram illustrating an example of the correlationinformation that the correlation information recorder 408 b records. InFIG. 5 , the vertical axis represents the light emission intensity andthe horizontal axis represents the degree of invasion (the depth and thearea) of living tissue by thermal treatment. In FIG. 5 , the straightline Ly represents the correlation between the light emission intensityand the degree of invasion (depth and area) to the living tissue bythermal treatment.

As represented by the straight line Ly in FIG. 5 , the larger the degreeof invasion of living tissue by thermal treatment is, the higher thelight emission intensity is.

Overview of Fluorescence Observation Mode

A fluorescence observation mode (thermal treatment observation mode)that is executable in the endoscope system 1 will be described next.FIG. 6 is a diagram schematically illustrating a principle ofobservation in the fluorescence observation mode.

As represented in the graph G11 in FIG. 6 , first of all, the controldevice 4 causes the second light source unit 403 to emit light, therebyapplying the excitation light (having a center wavelength of 415 nm) tothe living tissue O10 (thermal treatment area) resulting from thermaltreatment performed by the laser irradiation device 5 on the subject. Inthis case, as represented in the graph G12 in FIG. 6 , while thereflection light containing at least the components and the return lightthat is reflected on the living tissue O10 (the thermal treatment area)(simply referred to as “reflection light W10” below) is blocked by thecut filter 203 and the intensity of the reflection light lowers, part ofthe components on a side of wavelengths longer than that of thewavelength band that is mostly blocked is incident on the imaging device204 with its intensity not lowering.

More specifically, as represented in the graph G12 in FIG. 6 , the cutfilter 203 blocks most of the reflection light W10 that is incident onthe G pixels and that has a wavelength band of short wavelengthscontaining the wavelength band of the excitation light and transmits thewavelength band on a side of wavelengths longer than that of thewavelength band that is mostly blocked. Furthermore, as represented inthe graph G12 in FIG. 6 , the cut filter 203 transmits fluorescence(WF10) that is emitted by the self-luminous AGEs in the living tissueO10 (thermal treatment area). Thus, the reflected light W1 with thelower intensity and the fluorescence (WF10) are incident on each of theR pixels, the G pixels and the B pixels.

As represented by the polygonal chain L_(NG) of fluorescencecharacteristics in the graph G12 in FIG. 6 , the G pixels havesensitivity to fluorescence; however, because of a very small responseto fluorescence, the output value is a small value.

Thereafter, the image processor 406 acquires an imaging signal (RAWdata) from the imaging device 204 of the endoscope 2 and performs imageprocessing on each of the signal values of the G pixels and the B pixelscontained in the acquired imaging signal, thereby generating afluorescence image. In this case, the signal values of the G pixelscontain fluorescence information. The B pixels contain backgroundinformation from the living tissue of the subject containing the thermaltreatment area. In this case, the image processor 406 generates afluorescence image by performing demosaicing, processing of calculatinga ratio of intensities of the respective pixels, processing ofdetermining a fluorescence area and a background area, and imageprocessing using different parameters on each of color component signals(pixel values) of pixels positioned in the fluorescence area and each ofcolor component signals (pixel values) of pixels positioned in thebackground area. The image processor 406 outputs the fluorescence imageto the display device 3. The fluorescence area is an area where thefluorescence information is more dominant than the backgroundinformation. The background area is an area where the backgroundinformation is more dominant than the fluorescence information.Specifically, the extractor 406 b in the image processor 93 extracts thefluorescence area and the background area by, while determining that itis a fluorescence area when the intensity ratio of a reflection lightcomponent signal corresponding to the background information containedin the pixels and the fluorescence component signal corresponding to thefluorescence information is at or above a given threshold (for example,0.5 or more), determining that it is a background area when theintensity ratio is under the given threshold.

As described above, the fluorescence observation mode (thermal treatmentobservation mode) makes it possible to easily observe the living tissuethat is thermally treated by the laser irradiation device 5 (thermaltreatment area).

Overview of Normal Observation Mode

A normal light observation mode that is executable by the endoscopesystem 1 will be described next. FIG. 7 is a diagram schematicallyillustrating a principle of observation in the normal light observationmode.

As illustrated in FIG. 7 , first of all, the control device 4 causes thefirst light source unit 402 to emit light, thereby applying white lightW3 to the living tissue O10. In this case, part of reflection light andreturn light (simply referred to as “reflection light WR30, reflectionlight WG30, and reflection light WB30” below) that are reflected on theliving tissue O10 is blocked by the cut filter 203 and the rest isincident on the imaging device 204. Specifically, as illustrated in FIG.7 , the cut filter 203 blocks the reflection light having the wavelengthband of short wavelengths containing the wavelength band of narrow-bandlight. For this reason, as illustrated in FIG. 7 , the components oflight having the wavelength band of blue that is incident on the Bpixels are less than those in the case where the cut filter 203 is notarranged.

Thereafter, the image processor 406 acquires an imaging signal (RAWdata) from the imaging device 204 and performs image processing on eachof the signal values of the R pixels, the G pixels and the B pixelscontained in the acquired imaging signal, thereby generating anobservation image (white light image) that is a display image. In thiscase, because the blue components contained in the imaging signal areless than those in conventional white light observation, the imageprocessor 406 performs white balance adjustment processing of adjustingwhite balance such that the ratio of red components, green components,and blue components is constant.

As described above, the normal observation mode makes it possible toobserve a natural observation image (white image) even when the cutfilter 203 is arranged.

Manipulation Method of Flexible Transurethral Lithotomy Using EndoscopeSystem

A manipulation method of flexible transurethral lithotomy (f-TUL)performed by a practitioner using the endoscope system 1 will bedescribed next. FIG. 8 is a flowchart illustrating a manipulation methodof flexible transurethral lithotomy (f-TUL) performed by a practitionerusing the endoscope system 1. FIGS. 9A to 9F are diagrams illustratingtransition of the image that the display device 3 displays in flexibletransurethral lithotomy f-TUL).

As illustrated in FIG. 8 , first of all, the practitioner inserts theinsertion portion 21 of the endoscope 2 into the urinary tract (ureter)of a subject while applying white light (normal light) (step S1). Inthis case, as illustrated in FIG. 9A, the practitioner inserts theinsertion portion 21 of the endoscope 2 into the urinary tract of thesubject while observing an observation image P1 resulting from the whitelight and displayed on the display device 3.

Subsequently, while viewing the observation image P1 that is displayedon the display device 3, the practitioner checks a calculus that iscaused in the subject (step S2). In this case, as illustrated in FIG.9B, the practitioner checks the size and the position of a calculus K1while observing the observation image P2 displayed on the display device3 and inserting the insertion portion 21 of the endoscope 2 into theurinary tract of the subject and while searching for the calculus K1.

Thereafter, while viewing the observation image that is displayed on thedisplay device 3, the practitioner inserts the laser irradiation device5 into the ureter of the subject via the treatment tool insertionportion 222 of the endoscope 2 and applies a laser to the calculus (stepS3). In this case, as illustrated in FIG. 9C, the practitioner fragmentsthe calculus K1 by applying a laser with the laser irradiation device 5to the calculus K1 while observing the observation image P3 that isdisplayed on the display device 3.

Subsequently, while viewing the observation image that is displayed onthe display device 3, the practitioner extracts the fragmented calculusfrom the subject with a basket via the treatment tool insertion portion222 of the endoscope 2 (step S4). In this case, as illustrated in FIGS.9D and 9E, while viewing an observation image P4 or an observation imageP5, the practitioner extracts the fragmented calculus K1 that isfragmented with a treatment tool that can be gripped, for example, abasket catheter K2 via the treatment tool insertion portion 222 of theendoscope 2.

Thereafter, the practitioner switches the observation mode in which theendoscope 2 performs irradiation from the normal light observation modeto the fluorescence observation mode (a thermal treatment observationmode) by operating the operation unit of the endoscope 2 (step S5). Inthis case, by causing the second light source unit 403 to emit light,the control device 4 applies excitation light to the subject. Asillustrated in FIG. 9F, by observing a fluorescence area Ql contained ina fluorescence image P6 that is displayed on the display device 3, thepractitioner knows a degree of invasion by a burn caused by the laserirradiation device 5.

Subsequently, the practitioner knows the degree of invasion ofsurrounding tissue by a burn caused by the laser irradiation device 5while switching the observation mode of the endoscope 2 between thefluorescence observation mode and the normal light observation modealternately by operating the operation unit 22 of the endoscope 2 (stepS6).

Thereafter, while referring to whether it is necessary to place a stentand a placement period that the display device 3 displays, thepractitioner determines placement of a stent and a placement period(step S7). In this case, based on the light emission intensity of alight emission area contained in the fluorescence image P6, the controldevice 4 outputs placement period information on whether it is necessaryto place a stent in the urinary tract and on the placement period ontothe observation image that is displayed on the display device 3.Accordingly, with reference to the placement period information that isdisplayed on the display device 3, the practitioner determines whetherit is necessary to place a stent in the urinary tract and a placementperiod. Note that a method of making an estimation on whether it isnecessary to place a stent in the urinary tract and on a placementperiod that the control device 4 causes the display device 3 to displaywill be described below.

Subsequently, when a stent is placed in the urinary tract, thepractitioner places a stent in the urinary tract (step S8). Thereafter,the practitioner pulls the endoscope 2 out of the urinary tract of thesubject and ends manipulation.

As described, after fragmenting a calculus that is positioned in theurinary tract of a subject by a laser and extracting the fragmentedcalculus from the subject with a treatment tool, or the like, thepractitioner switches the observation mode of the endoscope system 1from the normal light observation mode to the fluorescence observationmode and knows the degree of invasion of living tissue of the subject bythe laser, thereby determining whether it is necessary to place a stentand a placement period.

Process Executed by Endoscope System

A process that the endoscope system 1 executes will be described next.

FIG. 10 is a flowchart illustrating an overview of the process executedby the endoscope system 1.

As illustrated in FIG. 10 , first of all, the controller 409 causes thefirst light source unit 402 to emit light by controlling the lightsource controller 404, thereby applying white light to a subject (stepS101).

Subsequently, the image processor 406 acquires an imaging signal fromthe imaging device 204 of the endoscope 2, generates an observationimage that is a display image, and outputs the observation image to thedisplay device 3 (step S102).

Thereafter, the controller 409 determines whether a change signal thatchanges the observation mode to the fluorescence observation mode isinput from the input unit 407 or the operation unit 22 of the endoscope2 (step S103). When the controller 409 determines that the change signalthat changes the observation mode to the fluorescence observation modeis input from the input unit 407 or the operation unit 22 of theendoscope 2 (YES at step S103), the endoscope system 1 moves to stepS104 described below. On the other hand, when the controller 409determines that the change signal that changes the observation mode tothe fluorescence observation mode is not input from the input unit 407or the operation unit 22 of the endoscope 2 (NO at step S103), theendoscope system 1 moves to step S120 described below.

At step S104, the controller 409 causes the second light source unit 403to emit excitation light by controlling the light source controller 404.

Subsequently, the image processor 406 generates a fluorescence imagebased on the imaging signal that is generated by the imaging device 204of the endoscope 2 (step S105).

Thereafter, the extractor 406 b extracts a fluorescence area that iscontained in the fluorescence image that is generated by the generator406 a (step S106). Specifically, the extractor 406 b extracts afluorescence area by performing the binary process, or the like, on thefluorescence image. When a plurality of fluorescence areas are containedin the fluorescence image, the extractor 406 b extracts the fluorescenceareas.

Subsequently, the calculator 406 c calculates a fluorescence intensityof the fluorescence area that is extracted by the extractor 406 b (stepS107). In this case, when the extractor 406 b extracts a plurality offluorescence areas, the calculator 406 c calculates fluorescenceintensities of the respective fluorescence areas.

Subsequently, the estimator 406 d determines whether there are aplurality of fluorescence areas (step S108). When the estimator 406 ddetermines that there are a plurality of fluorescence areas (YES at stepS108), the endoscope system 1 moves to step S109 described below. On theother hand, when the estimator 406 d determines that there are not aplurality of fluorescence areas (NO at step S108), the endoscope system1 moves to step S110 described below.

At step S109, the estimator 406 d makes an estimation on whether toplace a stent in the urinary tract based on the highest light emissionintensity among those of the fluorescence areas that are calculated bythe calculator 406 c and correlation information that the correlationinformation recorder 408 b records. Specifically, as illustrated in FIG.11 , the estimator 406 d makes an estimation on whether to place a stentin the urinary tract based on the correlation information that isrecorded in the correlation information recorder 408 b and the lightemission intensity that is calculated by the calculator 406 c. Forexample, as illustrated in FIG. 11 , the estimator 406 d determineswhether the light emission intensity is under a threshold TL1representing a value indicating that it is unnecessary to place a stentbased on the correlation information that is recorded by the correlationinformation recorder 408 b and the light emission intensity that iscalculated by the calculator 406 c and, when the light emissionintensity is under the threshold TL1, estimates that it is unnecessaryto place a stent in the urinary tract. On the other hand, when the lightemission intensity is at or above the threshold TL1 according to thecorrelation information that is recorded by the correlation informationrecorder 408 b and the light emission intensity that is calculated bythe calculator 406 c, the estimator 406 d estimates that it is necessaryto place a stent in the urinary tract. After step S109, the endoscopesystem 1 moves to step S111 described below.

At step S110, the estimator 406 d makes an estimation on whether it isnecessary to place a stent based on the light emission intensity that iscalculated by the calculator 406 c and the correlation information thatis recorded by the correlation information recorder 408 b. After stepS110, the endoscope system 1 moves to step S111 described below.

At step S111, when the estimator 406 d estimates placement of a stent inthe urinary tract (YES at step S111), the endoscope system 1 moves tostep S112 described below. On the other hand, when the estimator 406 destimates placement of no stent in the urinary tract (NO at step S111),the endoscope system 1 moves to step S114 described below.

At step S112, the estimator 406 d estimates a period during which astent is placed in the urinary tract based on the light emissionintensity that is calculated by the calculator 406 c and the correlationinformation that is recorded in the correlation information recorder 408b. Specifically, as illustrated in FIG. 12 , the estimator 406 destimates a period during which a stent is placed in the urinary tractbased on the light emission intensity that is calculated by thecalculator 406 c and the correlation information that is recorded by thecorrelation information recorder 408 b. For example, as illustrated inFIG. 12 , the estimator 406 d estimates that a placement period for astent is short when the correlation between the light emission intensityand the degree of invasion is positioned in a first zone Z1 (Low)according to the light emission intensity that is calculated by thecalculator 406 c and the correlation information that is recorded by thecorrelation information recorder 408 b, estimates that a placementperiod for a stent is normal when the correlation between the lightemission intensity and the degree of invasion is positioned in a secondzone Z2 (middle), and estimates that a placement period for a stent islong when the correlation between the light emission intensity and thedegree of invasion is positioned in a third zone Z3 (high). The shortperiod here is about few days, the normal period is about a week, andthe long period is 10 days or more.

Subsequently, the output unit 406 e outputs the placement periodinformation to the display device 3 (step S113). Specifically, asillustrated in FIG. 13 , the output unit 406 e outputs placement periodinformation M1 and an observation image P10 that is a display image tothe display device 3. The placement period information M1 contains m1 ofa maximum invasion degree (a maximum depth of invasion based on thefluorescence intensity), m2 on whether it is necessary to place a stent,and m3 of a preferred placement period for a stent.

Thereafter, the controller 409 determines whether a change signal thatchanges the observation mode to the normal light observation mode isinput from the input unit 407 or the operation unit 22 of the endoscope2 (step S114). When the controller 409 determines that the change signalthat changes the observation mode to the normal light observation modeis input from the input unit 407 or the operation unit 22 of theendoscope 2 (YES at step S114), the endoscope system 1 moves to stepS115 described below. On the other hand, when the controller 409determines that the change signal that changes the observation mode tothe normal light observation mode is not input from the input unit 407or the operation unit 22 of the endoscope 2 (NO at step S114), theendoscope system 1 returns to step S104 described above.

At step S115, the controller 409 causes the first light source unit 402to emit light by controlling the light source controller 404, therebycausing application of white light.

Subsequently, the image processor 406 acquires an imaging signal fromthe imaging device 204 of the endoscope 2, generates an observationimage, and outputs the observation image to the display device 3 (stepS116). Specifically, the generator 406 a acquires an imaging signal fromthe imaging device 204 and, based on the imaging signal, generates anobservation image.

Thereafter, the controller 409 determines whether there is invasion ofliving tissue by a laser according to the estimator 406 d (step S117).Specifically, the controller 409 determines whether the estimator 406 destimates placement of a stent in the urinary tract and, when theestimator 406 d estimates placement of a stent in the urinary tract,determines that there is invasion of living tissue by a laser. When thecontroller 409 determines that there is invasion of living tissue by alaser (YES at step S117), the endoscope system 1 moves to step S118described below. On the other hand, when the controller 409 determinesthat there is no invasion of living tissue by a laser (NO at step S117),the endoscope system 1 moves to step S119 described below.

At step S118, the output unit 406 e superimposes the placement periodinformation indicating whether it is necessary to place a stent and theplacement period onto the observation image that is generated by thegenerator 406 a and outputs the observation image with the placementperiod information superimposed thereon to the display device 3. Afterstep S118, the endoscope system 1 moves to step S120 described below.

At step S119, the output unit 406 e outputs the observation image thatis generated by the generator 406 a to the display device 3. After stepS119, the endoscope system 1 moves to step S120 described below.

At step S120, the controller 409 determines whether an end signal thatends observation of the subject is input from the input unit 407 or theoperation unit 22 of the endoscope 2. When the controller 409 determinesthat the end signal that ends observation of the subject is input fromthe input unit 407 or the operation unit 22 of the endoscope 2 (YES atstep S120), the endoscope system 1 ends the process. On the other hand,when the controller 409 determines that the end signal that endsobservation of the subject is not input from the input unit 407 or theoperation unit 22 of the endoscope 2 (NO at step S120), the endoscopesystem 1 returns to step S101 described above.

According to the embodiment described above, because the estimator 406 destimates a placement period during which the medical tool is placed ina lumen based on a fluorescence intensity that is calculated by thecalculator 406 c and the output unit 406 e outputs placement periodinformation on the placement period that is estimated by the estimator406 d and an observation image obtained by capturing an image of livingtissue to the display device 3, it is possible to objectively know theplacement period during which the medical tool is placed in a lumen.

According to the embodiment, because the estimator 406 d estimates adegree of invasion of living tissue with an energy device based on afluorescence intensity that is calculated by the calculator 406 c andestimates a placement period based on the degree of invasion, it ispossible to objectively know the placement period during which themedical tool is placed in a lumen.

According to the embodiment, because the estimator 406 d estimates adegree of invasion based on correlation information that is recorded bythe correlation information recorder 408 b and a fluorescence intensitythat is calculated by the calculator 406 c, it is possible to estimatean actual degree of invasion of living tissue.

According to the embodiment, because the estimator 406 d makes anestimation on whether to place the medical tool in a lumen based on afluorescence intensity that is calculated by the calculator 406 c, auser, such as a practitioner, is able to objectively know whether it isnecessary to place the medical tool in the lumen.

According to the embodiment, when a plurality of fluorescence areas areextracted by the extractor 406 b, because the estimator 406 d estimatesa placement period during which the medical tool is placed in a lumenbased on the highest fluorescence intensity among those of thefluorescence areas, it is possible to support the most appropriateinformation in the situation of the sequential treatment.

In the embodiment, the output unit 406 e outputs the observation imageand the placement period information to the display device 3. Forexample, the observation image with the placement period informationbeing superimposed thereon may be output to the display device 3.

In the embodiment, the output unit 406 e outputs the placement periodinformation to the display device 3; however, embodiments are notlimited to this, and for example, when the estimator 406 d estimatesinvasion of living tissue with the laser irradiation device 5, an outputindicating that invasion of living tissue is detected may be made.Specifically, as illustrated in FIG. 14 , when the estimator 406 destimates invasion of living tissue with the laser irradiation device 5,the output unit 406 e may output invasion information M10 indicatingthat invasion of living tissue is detected to the display device 3.Accordingly, the practitioner is able to know invasion of living tissuewith the laser irradiation device 5.

In the embodiment, the output unit 406 e outputs the placement periodinformation to the display device 3; however, embodiments are notlimited thereto and, for example, when the calculator 406 c calculateseach of light emission intensities of the fluorescence areas, an outputto the display device 3 may be made such that degrees of invasion of therespective fluorescence areas are identifiable. Specifically, asillustrated in FIG. 15 , the output unit 406 e may superimposes thedegrees of invasion on an observation image P30 in a display modecorresponding to the light emission intensities of the fluorescenceareas Q11 to Q13 that are calculated by the calculator 406 c and depthinformation M20 on invasion depths and the placement period informationM1 may be output together to the display device 3. In this case, theestimator 406 d estimates whether it is necessary to place a stent and aplacement period based on the light emission intensity of a fluorescencearea Q12 on a near-point side compared to that of the light emissionintensities of fluorescence areas Q11 and Q13 on a far-point side amongthe fluorescence areas Q11 to Q13 in an area H1 illustrated in FIG. 16 .As for a method of determining the far-point side and the near-pointside, the estimator 406 d determines whether luminance information oneach pixel of the observation image (white light image) is at or above agiven threshold, estimates that pixels at or above the given thresholdare on the near-point side and estimates that pixels under the giventhreshold are on the far-point side, thereby estimating the far-pointside and the near-point side. When the extractor 406 b detects afluorescence area on only the far-point side, the output unit 406 e maymake only an output indicating that invasion of living tissue with theenergy device is caused to the display device 3. Furthermore, when theextractor 406 b detects a fluorescence area on only the far-point side,the calculator 406 c may calculate a fluorescence intensity byperforming amplification using a gain, etc., on the signal values ofpixels positioned in the fluorescence area.

In the embodiment, the practitioner switches the observation mode byoperating the input unit 407 or the operation unit 22, thereby switchingbetween the observation image and the fluorescence image. Alternatively,a fluorescence image may be acquired by applying excitation light byautomatic switching at a given frame rate (for example, 60 fps) and atevery given frames (for example, 10 fps). In this case, while outputtingthe observation image to the display device 3, the output unit 406 e mayoutput invasion information M10 indicating that invasion of livingtissue is detected only when the estimator 406 d estimates a degree ofinvasion with the energy device.

In the embodiment, the estimator 406 d estimates whether it is necessaryto place a stent and estimates a placement period during which a stentis placed in the urinary tract; however, embodiments are not limitedthereto and, for example, the estimator 406 d may make only any one ofan estimation on whether it is necessary to place a stent and anestimation of a placement period during which a stent is placed in theurinary tract. Needless to say, the estimator 406 d may make only anyone of an estimation on whether it is necessary to place a stent and anestimation of a placement period during which a stent is placed in theurinary tract according to an operation on the input unit 407 or theoperation unit 22 by the practitioner.

Modification 1

In the embodiment, the transmission characteristics of the cut filter203 are changeable. FIG. 17 is a diagram schematically illustratingtransmission characteristics of a cut filter according to a modificationof the embodiment. In FIG. 17 , the horizontal axis represents thewavelength (nm) and the vertical axis represents the transmissioncharacteristics. In FIG. 17 , the polygonal chain L_(FF) representstransmission characteristics of a cut filter 203A, the polygonal chainL_(V) represents the wavelength characteristics of the excitation light,and the polygonal chain L_(NG) represents the wavelength characteristicsof fluorescence caused by application of the excitation light toadvanced glycation end products caused by thermal treatment on livingtissue performed by an energy device, for example, the laser applicationdevice 5.

As represented by the polygonal chain L_(V) and the polygonal chainL_(NG), the cut filter 203A transmits part of the excitation light thatis reflected from the living tissue of the observation area andtransmits only fluorescence components. Specifically, the cut filter203A blocks light having a wavelength band on a short-wavelength sidefrom 400 nm and under 430 nm containing the excitation light andtransmits light having a wavelength band on a long wavelength side from430 nm containing fluorescence caused by application of excitation lightto advanced glycation end products caused by thermal treatment.

OTHER EMBODIMENTS

It is possible to form various embodiments by appropriately combining aplurality of elements disclosed in the endoscope system according to theembodiment described above. For example, some elements may be omittedfrom the entire elements described with respect to the endoscope systemaccording to the embodiment described above. Furthermore, the elementsdescribed with respect to the endoscope system according to theembodiment described above may be combined as appropriate.

In the embodiment, the first light source unit, the second light sourceunit and the light source controller are provided integrally; however,embodiments are not limited thereto. For example, a light source deviceincluding the first light source unit, the second light source unit andthe light source controller and a control device may be providedindependently.

In the embodiment, the endoscope system uses a flexible endoscope;however, embodiments are not limited thereto, and an endoscope systemusing a rigid endoscope, a medical surgery robot using a plurality ofrigid endoscopes and a laser irradiation device, or a medicalobservation system is also usable.

In the endoscope system according to the embodiment, the “unit”, “-er”and “-or” described above may be read as “means”, “circuitry”, or thelike. For example, the controller may be read as a control means or acontrol circuitry.

In the description of the flowcharts herein, the context of the processamong steps is clearly specified using expressions including “first ofall”, “thereafter”, and “subsequently”; however, the order of theprocesses necessary to implement the disclosure is not uniquelydetermined by those expressions. In other words, the order of processesin the flowcharts described herein is changeable within a range withoutinconsistency.

Some embodiments of the present application have been described above indetail according to the drawings and the embodiments are exemplary and,starting from the modes described in the disclosure section, thedisclosure can be carried out in other modes on which variousmodifications and improvements are made according to the knowledge ofthose skilled in the art.

The disclosure can also employ the following manipulation:

(1) A manipulation method of flexible transurethral lithotomy using anendoscope system, the manipulation method comprising:

inserting an endoscope into a urinary tract of a subject by white lightobservation;

fragmenting a calculus that is positioned in the urinary tract byapplying a laser to the calculus;

extracting the calculus that is fragmented by the laser from the urinarytract;

switching an observation method performed by the endoscope tofluorescence observation;

internally observing the subject by fluorescence observation; and

placing a stent after observation by the fluorescence observation.

According to the disclosure, an effect that it is possible toobjectively know a placement period during which a medical tool isplaced in a lumen is achieved.

What is claimed is:
 1. An assist device comprising: a processorconfigured to generate a fluorescence image based on an imaging signalthat is generated by capturing an image of fluorescence caused byexcitation light that is applied to a living tissue, calculate afluorescence intensity based on the fluorescence image, based on thefluorescence intensity, estimate a placement period during which amedical tool is placed in a lumen, and output placement periodinformation on the placement period and an observation image obtained bycapturing an image of the living tissue.
 2. The assist device accordingto claim 1, wherein the processor is configured to estimate a degree ofinvasion of the living tissue with an energy device based on thefluorescence intensity and estimate the placement period based on thedegree of invasion.
 3. The assist device according to claim 2, whereinthe processor is configured to estimate the degree of invasion based onthe fluorescence intensity and correlation information representing acorrelation between a degree of invasion and a fluorescence intensitythat are previously measured.
 4. The assist device according to claim 1,wherein the processor is configured to make an estimation on whether toplace the medical tool in the lumen.
 5. The assist device according toclaim 1, wherein the processor is configured to superimpose theplacement period information onto the observation image and output theobservation image with the placement period information superimposedthereon.
 6. The assist device according to claim 1, wherein theprocessor is configured to extract a fluorescence area from thefluorescence image and, when a plurality of fluorescence areas areextracted, estimate the placement period based on a highest fluorescenceintensity.
 7. The assist device according to claim 6, wherein theimaging signal is obtained by capturing an image of a urinary tract thatextends in a depth direction, and the processor is configured to, whenthe plurality of fluorescence areas are extracted, estimate theplacement period based on the fluorescence intensity of the fluorescencearea that is positioned on a side of a near point that is the closest toan imaging optical system.
 8. The assist device according to claim 6,wherein the processor is configured to, when the plurality offluorescence areas are extracted, output the plurality of fluorescenceareas such that each of the plurality of fluorescence areas isidentifiable based on the fluorescence intensity of each of theplurality of fluorescence areas.
 9. The assist device according to claim1, wherein the medical tool is any one of a stent, a catheter and anindwelling needle.
 10. The assist device according to claim 1, whereinthe lumen is a urinary tract.
 11. The assist device according to claim1, wherein the excitation light has a wavelength band from 390 nm to 430nm, the fluorescence has a wavelength band from 500 nm to 640 nm, andthe imaging signal is obtained by capturing an image of transmissionlight that passes through a cut filter that blocks light on a side ofshort wavelengths from the 430 nm.
 12. An endoscope system comprising:an endoscope that is insertable into a lumen of a subject; a lightsource configured to emit excitation light that excites advancedglycation end products caused by performing thermal treatment on aliving tissue; and a control device that is detachable from theendoscope, the endoscope including an imaging device configured togenerate an imaging signal by capturing an image of fluorescence causedby the excitation light, and a cut filter that is provided on a side ofa light receiving surface of the imaging device, the cut filter beingconfigured to block light on a side of short wavelengths containing partof a wavelength band of the excitation light, the control deviceincluding an assist device that assists a practitioner, and the assistdevise including a processor configured to generate a fluorescence imagebased on the imaging signal, calculate a fluorescence intensity based onthe fluorescence image, based on the fluorescence intensity, estimate aplacement period during which a medical tool is placed in the lumen, andoutput placement period information on the placement period and anobservation image obtained by capturing an image of the living tissue.13. An assist method executed by an assist device, the methodcomprising: generating a fluorescence image based on an imaging signalthat is generated by capturing an image of fluorescence caused byexcitation light that is applied to a living tissue; calculating afluorescence intensity based on the fluorescence image; based on thefluorescence intensity, estimating a placement period during which amedical tool is placed in a lumen; and outputting placement periodinformation on the placement period and an observation image obtained bycapturing an image of the living tissue.
 14. A non-transitorycomputer-readable recording medium with an executable program storedthereon, the program causing an assist device to execute generating afluorescence image based on an imaging signal that is generated bycapturing an image of fluorescence caused by excitation light that isapplied to a living tissue; calculating a fluorescence intensity basedon the fluorescence image; based on the fluorescence intensity,estimating a placement period during which a medical tool is placed in alumen; and outputting placement period information on the placementperiod and an observation image obtained by capturing an image of theliving tissue.